Computer system and method for detecting, extracting, weighing, benchmarking, scoring, reporting and capitalizing on complex risks found in buy/sell transactional agreements, financing agreements and research documents

ABSTRACT

Computer-implemented systems and methods enhance a user’s sophistication as she/he reviews complex information sources using specialized detective tools provided by a user interface of the computer system. The specialized investigative inquiries are stored in a database and are particularly tailored a priori by a subject-matter content designer for the type of documents being reviewed for risk and opportunity. The investigative scripts are organized into to a path of risk-related subjects or topics, and within each path of subjects/topics the investigative scripts are organized into a specialized inquiry or flow chart.

PRIORITY CLAIM

The present application is a continuation of U.S. Application Serial No.17/553,949, filed Dec. 17, 2021, now U.S. Pat. 11,688,017, issued Jun.27, 2023, which is a continuation of U.S. Application Serial No.17/061,650, filed Oct. 2, 2020, now U.S. Pat. 11,205,233, issued Dec.21, 2021, which claims priority to U.S. provisional patent applicationSerial No. 62/909,494, filed Oct. 2, 2019, with the same title andinventor as stated above, and which is incorporated herein by referencein its entirety.

BACKGROUND

The written agreements (contracts) that document a buy/sell transactionand financing agreement are usually bespoke, but not always. Thesecontracts can be applicable to equity, debt, derivative and otherfinancial oriented transactions. As the financial issue at hand grows invalue, complexity and risk, the documents tend to get longer and morecomplex. Some transactional agreements can include more than one millionpages of contract related documents. Through a quick reference toanother document or web site, that document or web site could include amillion pages of content as well. Some contracts have two parties whileother contracts could have many parties. Although all parties to anindividual transaction matter tend to review the contract document foraccuracy, the contracts are usually drafted initially by one party whoseeks to bias the contractual rights and privileges to their side. Thisis often achieved by scribing significant deal points in a descriptiveor prescriptive fashion. As a further commercial tactic, however, thecontract terms often include (1) intentionally conflicting language in alater section, (2) “collars” on when certain provisions are in-forceand/or (3) intentional drafting omissions (that can be used laterfavorably in the event of a future dispute). These drafting complexitiescan arise at multiple levels; at the transaction level, the specifiedasset level, at the borrowing entity level, at the borrowing entity’ssponsor level, at the parent company level, etc. Some transactions caninclude or reference millions pages of contract documents, thousands ofInternet links, hundreds of references to third party items, audio orvideo interviews with numerous people, call center transcripts,distributed ledge data elements etc. The scale of the informationsources related to a transactional or financial matter can be massive,found in many locations and may change over time. When they are coupledwith the complexities of the contractual arrangement, there can be tensof millions of risk concerns or reward opportunities.

Sometimes the parties only have a limited time, such as 48 hours, tomake a lending / investing decision to participate in a transaction ornot. This requires them to analyze risk associated with the transactionand use risk analysis intuition to decide if they wish to participate inthe transaction. Detailed analysis often leads to more informeddecision-making.

On a buy / sell transaction, the contract and regulatory documents andfilings to memorialize that buy/sell or financing agreement could relateto a complex global mega-mergers all the way down to local receiptsexchanged at the grocery counter.

On the financing side, a corporation or individual may borrow funds on asecured or unsecured basis. The agreements to evidence that borrowingcould be extremely dense and take different forms; loan, receivablecontract, custom IOU, bond etc.

On the financing side and away from a government, corporation or personfinancing a specific asset, there is a specialized financing tool whenlarge amount of loans or receivables are bundled and financed as acollective whole. In this embodiment, the parties bundle largequantities of loans and sell them in bulk to others to frequently book again or loss on that sale or to a special purpose trust who securitizesthem. Securitization often involves large pools of debt (IOU)instruments and is one type of financing transaction that typically hascomplex contracts and documentation. Securitization is the financialpractice of pooling various types of contractual debt instruments, suchas mortgages, unsecured marketplace debt instruments, auto loans orcredit card debt obligations. In that practice, they sell theirownership rights (including the related cash flows) to a third party whothen finances that purchase through the issuance of securities toinvestors, which may be in the form of bonds, pass-through securities,participations or collateralized debt/loan obligations (CDOs) to name afew of the options. Investors are repaid from the principal and interestcash flows collected from the underlying debt instruments andredistributed through the capital structure of the new financingvehicle. Securities backed by mortgage receivables are calledmortgage-backed securities (MBS), while those backed by other types ofreceivables are asset-backed securities (ABS) to name a select few.

A typical securitization process flow, in this case for MBS, is shown inFIG. 1 . The Borrower is an individual or organization which obtains aloan, in this case a mortgage, from a financial institution/lender(e.g., a bank) and pays the monthly payments on the loan for the fullterm of the loan. Sometimes, a Mortgage Broker acts as a facilitatorbetween a borrower and the lender. The financial institution/lender thensells the loan assets to the Issuer, which often includes transactionalcontracts. The Issuer typically is a bankruptcy-remote Special PurposeEntity (SPE) formed to facilitate the securitization financing processand to issue the securities to the investors. The Issuer then sells thesecurities as bonds, etc. to the Investors. The securities are backed bythe borrowers’ loans with the monthly payments on the securities fundedfrom the monthly payments from the borrowers on the loans.

The securitization often has multiple parties that work together overthe term of a securitization to operate the SPE. As an example, theServicer is an entity responsible for collecting the loan payments fromthe borrowers and for remitting these payments to the issuer for furtherdistribution to the investors. The servicer is generally obligated tomaximize the payments from the borrowers to the issuer, and isresponsible for handling performing as well as delinquent loans and therelated foreclosures and repossessions. The Trustee is a third partyappointed to represent the investors’ interests in the securitization.The trustee undertakes specified duties to help ensure that thesecuritization operates as set forth in the securitization documents,which may include determinations about the servicer’s compliance withestablished servicing criteria. Unfortunately, many SPEs do not includea senior party to oversee the ongoing administration of the SPE. Thisintroduces many risks that can materially impact an investment’s cashflow and in turn value. There are usually many more parties than justthe Trustee and servicer.

There are many information sources or data items (such as agreements)included in such a securitization. These data items could include theborrowers’ loan documents, the purchase, sale and/or pledge documentsbetween the lender(s) and the issuer/SPV that transfer the rights to theongoing payments from the borrowers to the issuer/SPV. There is alsotypically a Pooling and Servicing Agreement (PSA) and/or other operatingagreements that define how secured assets such as loans are combined inthe securitization, the administration and servicing of the loans,representations and warranties, and permissible loss mitigationstrategies that the servicer can perform in event of loan default. TheUnderwriter administers the placement of the securities to investors.Finally, the securitization transaction may include credit enhancement(designed to decrease the credit risk of the various parts of thestructure) provided by an independent third party, such as the CreditEnhancement Provider, in the form of letters of credit or guarantees.Some of these documents include “risk” disclosure sections that can bemore than 100 pages. A securitization with 10,000 underlying loans thatact as collateral could have 1 million + pages of documents.

Each securitization may include risk, such as the borrower not makingthe loan payments, and the asset backing a loan becoming impaired, etc.The securitization documents are supposed to explain the parties’ rolesin the event of risk contingencies, particularly the service provider’sroles. However, the securitization documents may not adequately addressthe risk contingencies and/or introduce new risks due to inconsistent oromitted contractual provisions that must be applied to the SPE until thelast loan is removed from the SPE. Inadequate securitization documentscan materially increase the cash flow variability and loss severity forthe investors and increase their secondary price volatility.

Corporate debt is another type of financing instrument. When a companyborrows money outright or pledges an asset to facilitate a financing,the details of the pledged asset becomes important. An asset could startduring a construction or manufacturing period, a user stabilizationperiod or a long-term usage period. The documents related to thisevolution can be voluminous and risks and rewards change as theborrowing company and the asset matures.

The information sources to memorialize the various aspects of thetransaction are not static. There are numerous instances that documents,filings, verbal agreements etc. can be amended, restated, altered,exchanged etc. The initial sources are often referred to as the originaldocuments, executed documents or final (static) documents. Reviseddocuments are sometimes called amended, waived or restated documents.They tend to be dynamic. While some of the data can be consideredstructured, most of the agreement contract matters are unstructured.

All of these risks are considered in relation to local and global marketissues. Weighing the static and dynamic features of the transactionrisks individually and collectively against a dynamic basket oftechnical markets benefit from intuition and using the assessment ofmaterial drivers of risk / reward.

Extracting data points from the underlying transaction and financialdocuments and other sources can be helpful. Extracting the data elementin a more specialized process that imbeds contextual application to theindividual risks at hand and how they stack together is particularlydesirable. When this contextual extraction is applied to millions ofdata items such as documents, filings, interviews, revised secured-assettapes etc., that are not well linked, the analytical leverage that isafforded to a person can be material. The value of this information andbenchmarking leverage results in a more informed analysis and investmentdecision. Smart well-researched analysis can result in trillions ofdollars of incremental value.

Risk is often used when assessing downside exposures. Although this isoften true, a risk element can sometimes be a positive attribute. Forinstance, a borrower event related to a risk element can occur which cangive a lender an opportunity to increase its lending rate. A risky itemcan be a profitable item from the lender’s perspective. This agreementto increase the rate in the event that a risk even occurs is oftenscribed in contractual documents.

The finance industry currently lacks helpful standards on how todocument and in turn analyze business transactions and financing dataelements including in the related documents.

Scale and a lack of standardization is a significant constraint someparties can have thousands of investments in these bespoke transactions.Having a way to quickly, consistently and insightfully weigh risks andenter those risk assessments into other predictive and maintenancemodels is a need to the lending and investment communities.

SUMMARY

This present invention improves upon existing computer systems thatreview documents in that a computer system according to the presentinvention, in various embodiments, empowers a person to better analyzerisk, measure risk, report important risk, mitigate risk and optimizethe investment opportunities associated with a risky item. Stillfurther, as the computer system analyzes more transaction/financingrelated matters, the computer system becomes more sophisticated andhelpful to the user, thereby enhancing its economic value with continuedusage.

While the contextual extraction and analysis of information is afoundational aspect of this invention, it is the sequencing of theextraction protocols that can result in an intuitive risk assessment.The investigative nodes are connected in a neural-like network that isset up to “connect the dots” as it relates to analyzing the materialrisk attributes of a transaction / financing deal. Further, as each nodeof sequenced and specialized inquiries is improved (as the specialistuses the system) the intuition features of this invention improve.

In one general aspect, the present invention consists of two primaryattributes; (1) a computerized database of specialized detection,contextual extraction, scoring, benchmarking intuition weighing andspecialized reporting tool and (2) specialized technology to perform acomplex review of key risk drivers as well as other elements using thosetools more precisely, more consistently, faster and in a lower costfashion. As more and more risk analysis reviews are performed using thecombined attributes of this invention, the computer systems of thepresent invention can store the discovered evidence (anonymized or not)and use it to enhance the sophistication and speed of a risk assessmentfor future users. Each risk analysis module is further weighed to createa neural analysis that creates “risk intuition”.

Risks can come in many forms but they rarely stand alone. Layered riskis a term used to identify the materiality and interdependence of onetransactional / financing feature against another.

The users of this risk-analyzing output can be, but are not limited to,transaction parties and non-parties, such as investors, governmentagencies, lawyers risk managers, compliance managers and researchers, toname a few. As the inventive tools are used more frequently, the systemcan create a growing form of synthetic institutional knowledge relatedto minimizing / optimizing the risks associated with the portfolio oftransactions reviewed to date. The computerized institutional knowledgerelated to assessing risk and making the output deliverable easy for aninvestor to use is made available using numerous digital delivery tools.

The inventive tool provides an investigating analyst with sophisticatedinvestigative expertise and risk weighting that is often found withseasoned veterans. This invention is a way to reduce key man risks andlower labor costs in the lending / investment fields.

Priming the pump. In early uses of the computer system, an individualperson will use each investigative node of the investigation framework,described further below, and coincidently train the computer to generatemore useful results in future applications. This initial task will“train” the invention to target its activities to matters related to thespecific item that is being analyzed. Effectively, this priming willstate tell the system “Where to find it”. Over time, this upfront workwill be enhanced with additional training that is implicitly found whenan analyst uses the invention and sites “Where I found it”.

The computer system can be pre loaded with controls. For example, atransactional matter that is being analyzed can be compared against acontrol item and thus establish a benchmark weighting relative to thatcontrol. This comparison against the control could be named a “Score,”but there are numerous other control comparison types including material/ immaterial, pass / fail, good/bad, etc. that could be used.

The contextual extraction of risk elements from many data sources suchas contract documents can be exportable to and used by external softwarewith related databases. The computer system’s automation of analysis andrisk weightings coupled with or exported to other databases canexponentially expand the sophistication of risk minimization /optimization and in turn economic value.

Information related to the corporate and financial industry tends to befound in eight primary areas. This includes documents, ledgers, audiocontent, visual content, filings, searchable databases, personalinterviews, research and other related media. In various embodiments,the computer system of the present invention views references and notesall of them. The term “documents” is used herein to denote all of thesetypes of information sources unless otherwise noted.

Access to the various data sources can be restricted for varioustransactions. For instance, a consumer may not wish to have the generalpublic know about their purchasing activities. The data elements relatedto a transaction are usually viewable by two groups of parties: insiderparties or the general public. These viewing exclusions / restrictionsor viewing authorizations also could restrict the population ofinformation sources (documents) that can be or cannot be analyzed andwho can read the risk assessment analysis. For confidentialitymanagement purposes, the “detective” aspect of the computer system canbe constrained based on the access privilege of the specialized analystand the distribution protocols of the allowable data, the specializedinvestigator and the report reviewer. An analysis based on more detailedprivate information will often generate a more insightful analysis. Ananalysis based on public information only which tends to be more of adisclosure document that has been biased by the writer of that summarypublic document. This public only restriction can reduce the depth ofthis invention’s analysis sophistication and introduces undesirable biasin the invention’s output due to bias of the input. Controlling who canread certain information sources and who can read the analysis output isan important part of the computer system. Limiting or constraining theinformation sources or viewers may be a confidentiality requirement.

Further complicating the risk analysis is that the transaction /financing related documents are often amended and restated, and contractprovisions are often waived over the course of the agreement’s term. Thecomputer system of the present invention can be applied to reviewing thedocuments upon the closing of the transaction (upfront staticinformation) or at a user-specified date after the transaction occurs orstarts. That specified date can be during the scheduled transaction’slife or even after the transaction has expired but follow-on risksremain or arise. The later analysis can incorporate new structured andunstructured data related to the transactional matter that collectivelymakes the analysis more timely and valuable. This evolving dynamic dataset could impact the revised weighing of the risk benchmarking andintron scoring.

In various embodiments, the privileged access output from the computersystem can comprise: (a) a computer generated report that identifies andbenchmarks / scores notable, positive and negative risk information; and(b) a data set comprised of identified evidence that supports thescoring and benchmarking practices. While the inventive computer systemdoes not, and is not intended to, predict the future, instead beingfocused more on the analysis of downside risk exposures if a downsideevent occurs. The computer system may have internal predictive abilitiesif programmed, however, such as most probable, less probable, leastprobable lending / investment risk elements. The system output coupledwith data items and other systems broadly usable in the finance industrycan yield advanced predictive modeling. Further, while the individualrisk analysis components of the computer system create a cumulativebenefit, they can be used individually in various embodiments. Thepredictive confidence of an investor’s purchase / sale of a financialinstrument related to a transactional matter should be materially higherbased on these sophisticated analysis processes.

The sophistication of an analysis paths within this invention canbenefit from extensive research on historical drivers of risk. Thisincludes research on legal complaints, judicial decisions, traderexperiences, changing regulations and global economic items that canimpact an investment’s forecasted risk/return profile, to name a few.

In one general aspect, therefore, the present invention is directed tocomputer-implemented systems and methods to enhance a user’ssophistication (e.g., a specialized investigator) as she/he reviewscomplex information sources (documents) and other data items relative toa particular issue(s), such as operational, reputational, market,transaction structuring, legal or financial risk, using specializeddetective tools provided by a user interface of the computer system. Theinformation sources such as documents can be related to transaction /financing related documents such as agreements and other documents thatare used in a financial arrangement, such as but not limited to asecuritization transaction, although the embodiments of the presentinvention could be used for other types of complex transactiondocuments. The specialized investigative inquiries are stored in adatabase and are particularly tailored a priori by a subject-mattercontent designer for the type of documents being reviewed for risk andopportunity. Further, the investigative scripts are organized into to apath of risk-related subjects or topics, and within each path ofsubjects/topics the investigative scripts are organized into aspecialized inquiry or flow chart. As such, when the specializedinvestigator selects one of the pre-populated specialized-responseoptions for a specialized inquiry or adds a custom response, the nextspecialized inquiry will depend on the prior specialized-responseoption, such that the specialized-response options lead the specializedinvestigator down a particular branch of the investigative specializedinquiry scripts. When the specialized investigator completes thesequence of individual inquiries for the evolving investigative branch,the computer system computes a benchmarking score, e.g., an assignedrisk score, for the particular subject of inquiry, where the score isbased on (e.g., assigned a priori by the subject-matter contentdesigner) that particular branch of the specialized inquiry. That scoreis often based on the system designer’s prior assessment of stacked orcumulative risk items plus user based additions. The specializedinvestigator(s) can provide responsive answers for each investigativescript for each subject or allow the specialized analysts to providetheir own. The assigned risk scores for each subject that are providedat the end of the survey can then be combined (e.g., averaged) to form acomposite assigned risk score for the transaction itself.

Insightful risk assessment is not always a single path dependentanalysis approach. It is the analysis of the variously risk assessmentsconnected in a neural fashion that can collectively generate anexceptional analysis output. This invention provide the topic levelscoring so that the collection of scores on many analysis items can beassessed in a cumulative fashion.

The specialized investigators’ responses to the inquiries also has theability to include guidance from the system designer about where in thedata in response to the specialized-response option is often found(e.g., a location identifier). Further, before proceeding to the nextinquiry, the analyst can be given input ability to identify where she/hein fact found informed information as well as record why the material ordescription at the location identifier found supports thespecialized-response. Additionally, the specialized analyst can recordseparate information sources that have conflicting, collaring, diluting,limiting or accelerating information. The specialized investigators’specialized-response and explanations for each specialized inquiry arestored in a database and indexed to the data source such as a documentthat was reviewed. The data found within a data item such as a documentcan also include the type of transaction (e.g., a securitization forMBS), the party (is) involved (e.g., the borrower servicer, issuer,lender, etc.), and the date, for example. All this information is storedsuch that when a specialized investigator reviews another document usingthe system in the future, the specialized-response option database canbe searched for similar transaction / financing related documents, withsimilar parties, etc., to determine the most likely location of wherethe supporting or contradictory evidence for the specialized-responseoption for investigative scripts can be found. In that way, when aspecialized investigator is providing responsive information on aspecialized inquiry for a particular transaction / financing relateddocument, the user interface can display to the specialized investigatorthe mostly likely place(s) where the specialized-response option may befound, based on the upfront set up of the inquiry by the system contentdesigner or from prior responses for the same investigative script andthe data access restrictions. The computer system of the presentinvention can provide the content designer with ability to pre load arecommended place to find supporting and possibly contradictory data inthe documents and other data items. The invention may further providethe specialized investigators with features to add location options tothat list of probable places to find responsive affirmative orconflicting data. Collectively, the invention allows future users tobenefit from the initial guidance and continuously updated guidance onwhere to find the evidence. This makes the future users faster infinding supporting evidence and more insightful.

An individual inquiry can programmatically incorporate a scored index orlist of prospective evidence citations to aid the specializedinvestigator. Further, the index/list of prospective evidence citationscan accumulate insight as more analysis is performed on moretransactions and the specialized investigator confirms where she/hefound the responsive evidence. Searching for “not found” items can beequally valuable as confirmatory evidence, excluded evidence orconflicting evidence found. Common business points that are notdocumented can introduce risk.

Still further, there may be instances where a specialized inquiry orspecialized-response option needs to be added. The inventive systemincludes communication tools that allows the specialized investigator toprovide feedback and make recommendations to her/his supervisor. Thesystem’s content designer can make “on the fly” or future enhancementsto the specialized inquiries and/or specialized-response options. Overthe passage of time, the need for enhancements is likely to be reducedas more transactions are analyzed. The data inclusions and reportreading exclusions mentioned previously will impact this as well.

The computer system may employ OCR tools to reduce human interface. Tobe the most effective, the OCR benefits from a list of guidance on whereto find affirmative or conflicting information. This inventions initialrecommendations on where to find responsive information plus the addedlocation information provided by the specialist provides highlysophisticated guidance for the OCR tool to find the most beneficialsupporting evidence. Further, the OCR tool may become more effective asit is used more often.

The population of data items that can be analyzed on a financial matteris unique to the analysis project at hand. The structured andunstructured data sources considered by the inventive system caninclude, in various embodiments, (1) draft and executed documents, (2)voice content, (3) video content, (4) electronic communications such ase mail, text, chat room and other social media, (5) regulatory or otherfilings, (6) complaints, (7) activity testing results, (8) alpha andnumeric data tape, (9) surveys, (10) transcripts, (11), researchreports, (12.) distributed ledger information and other digital orphysical media. Those items can be found using a storage tool so thatthe user can systematically find, mark and/or store the data. In thatconnection, embodiments of the present invention can be used for manydifferent types of transactions, even though each is unique, because thesubject-matter content designer coupled with or without the analyst’ssupervisor can tailor the investigative scripts to the relevant issuesinvolved in the transaction. For example, the system can be used toanalyze financing documents regardless of whether they are structuredcash or derivatives, whole loans or participating loans, secured orunsecured, guaranteed or unguaranteed, etc. If a particular transaction/ financing related document is more unique than others previouslyreviewed, the investigative scripts and risk analysis framework for itsmost similar previously reviewed transaction can be used as a startingplace. In that connection, agreements that can be reviewed with such asystem include, but are not limited to, loan agreements, transactingloan agreements, financing agreements related to portfolios of loans andother debt products, such as cash instruments, derivative instrumentsand insurance instruments. This library of data sources such asdocuments can also be linked to OCR resources and specialized datasorting tools to improve the speed and depth of analysis.

Given the analytical considerations to various and inconsistent datatypes and different source locations of data of affirmative andconflicting information as well as the fact that the extracted andanalyzed data is stored in a structured fashion, the computer system ofthe present invention can implicitly create large data sets ofstructured data that can be used elsewhere. Seeing that an analysis canbe performed numerous times on a matter over the course of that matter’suseful period, this recurring analysis and the storage of the applicableresponses can create a substantial database of standardizationtransactional / financial data features on one transaction. Theplacement of the raw or underlying information into the structureddatabase effectively creates data standards that can be used elsewhere.

The benchmarking feature of the invention grows in sophistication asmore transactions / financing matters are analyzed and the results aresaved and compared. Peer relative value analysis is an output from theinvention.

The computer system can employ dictionaries for one or more languages,such as English, to search for words in those languages.

The computer system of the present invention can be used as aninstructional tool for new persons entering the industry and wanting tolearn key items in significant detail. It can also be used by lowerlevel analysts in the instance that more seasoned personnel leave thefirm. Due to its accumulated knowledge, the computer system of thepresent invention can create conceptual digital institutional memory andreduces key man risk.

The sequence of investigation nodes can be structures with macroanalysis issues upfront and more detailed analysis issues later. Thisinvestigative sequencing can reduce time allocations. If the earlyanalysis results in no or not applicable, there may be limited value incontinuing on to a more detailed analysis. Descriptive items tend to beupfront and prescriptive items follow. The sequencing can vastly improvereliability and effectiveness.

The user of this tool can be a loan officer that seeks to create aninformed investment memo for the investment committee, a collectionsdepartment staffer that seeks to collect all monies on complex matters,risk supervisory managers, compliance supervisory managers, regulatorymanagers, researchers etc.

As the system is used for different transactions and accumulatestransactional knowledge (data points), it can become more insightful onone or more investigative nodes such that it can answer one or moreinquiries collectively or automatically. This automation can speed upthe time to conclude an analysis which results in lower cost and moretimely output.

The assigned risk scores can be used to identity mitigating actions toreduce or capitalize on the frequency of loss and loss severity. Forexample, if a particular transaction / financing related document has ascore that indicates a high risk, the individual subject matter assignedrisk scores can be reviewed to see which subject scores are lower thanexpected. Then the specialized-response options to the investigativescripts within the identified high-risk categories can be examined toidentify the risks at a more granular level and what mitigating actionscould be taken to reduce the risks that have been identified.

In addition to simplifying and enhancing the sophistication of thereview process for the specialized investigators, the focusedspecialized inquiries and associated specialized inquiry paths tend tolead to a better overall and more accurate assessment of the complexrisk elements and their related documents and other information.Further, by storing and analyzing the specialized investigators’specialized-response options and comments, system designers can adaptthe specialized inquiries and scoring (specialized inquiry) paths overtime to improve the overall risk assessments and benchmarking. Thesophistication of analysis and benchmarking can be further enhanced bythe content designers based on external research. These and otherbenefits of the present invention will be apparent from the descriptionthat follows.

In instances where a transactional and financing matter has a new anddifferent structures, the computer system’s accumulated knowledge may beless insightful on this structure, but the content designer can updatethe computer system as needed. This creates a second and more insightfulversion that later specialists can use when that structure arises againin the future. The content to inform the designer to consider an updatecan come from two locations; content found independently by the contentdesigner or “notes to designer” links that the specialist can provide tothe designer if they see fit. Collectively, this computerized feedbackloop makes the intellectual content within the system more insightfulover time.

The system, in various embodiments, incorporates multiple custom UIssuch as: a system’s software architect’s UI, content designer’s UI, datamanager’s UI, base detective content library UI, client-specificdetective content library UI, OCR management tool UI, Risk/rewardintuition weighing UI, supervisor assignment UI, specialized analyst UI,Feedback loop UI, Client / subscriber UI, findings export UI, and areport generator UI to name a few. Each custom UI can have specializedsecurity access, storage usage and supervisory tools.

The system can incorporate and be implemented with software modules thatare based on and/or written in many computer languages. The system canuse different languages for specialized tasks. The system can create aprotocol to capitalize on and coordinate their individual strengths.

Identifying risk is not all about downside exposures. A borrower’s riskcan be a lenders justification to charge more interest. This systemidentifies those risks and facilitates lending and investmentoptimization.

Finance is highly complex but tends to follow patterns. Like a reservoirdam, all transactions have “leaks”. Those leaks can become catastrophicif they are concentrated in a small area or influenced suddenly byoutside forces. Having the ability to detect outlier risks and assemblethem into usable forms is invaluable in a multi trillion-dollar market.The system’s computerized process of identifying, benchmarking andcontinuously improving risk analysis effectively creates digitizedinstitutional knowledge which in turn can be exported to other computersystems to initiate remedies to those risks. The way the system exportsthose items is a meaningful part of this computer system’s value.

FIGURES

Various embodiments of the present invention are described herein by wayof example in connection with the following figures, wherein:

FIG. 1 is a diagram of a typical securitization process flow;

FIG. 2 is a block diagram of a computer system according to variousembodiments of the present invention;

FIGS. 3, 4, 6 and 7 show screen shots that are displayable to anspecialized investigator reviewing a transaction / financing relateddocument according to various embodiments of the present invention;

FIG. 5 illustrates a database structure for storing responses toinvestigative scripts for transaction / financing related documentsaccording to various embodiments of the present invention;

FIG. 8 is a screen shot displayable to a programmer and/or subjectmatter specialized investigator in designing investigative scriptsaccording to various embodiments of the present invention;

FIG. 9 illustrates an example chapter report according to variousembodiments of the present invention;

FIG. 10 is a flow chart illustrating a process performed by the scoringsystem of FIG. 2 according to various embodiments of the presentinvention; and

FIGS. 11-19 are additional screen shots displayable by the systemaccording to various embodiments of the present invention.

DESCRIPTION

In one general aspect, the present invention is directed to computersystems and related computer-implemented methods for analysis-basedscoring of business risks and, in particular, providing graphical userinterfaces that make the analysis significantly more efficient thanexisting analysis techniques. For example, the systems and methods canbe used for benchmarking and in turn scoring the risks present intransactions and financing transactions for a securitization, such asloan and/or bond agreements. The computer system provides a userinterface to a user with investigative risk-oriented scripts tailored tothe type of subject and document the user is reviewing, such as asecuritization document or other type of document. As such, one user(e.g., an analyst) of the system may be a specialized investigatortasked with reviewing information sources such as documents and otherdata elements. The investigative scripts are designed to determine,validate or verify how the document addresses, or not, various riskcontingencies (such as operating, legal, market and/or royalty risk) orother issues described within by the document. Preferably, alongside theuser interface, an analyst reviews the data items such as the documentbeing scored to compare it against specialized-response options for theinvestigative scripts. The queries and corresponding response-optionsfor each query can be prepared by a subject matter expert or “designer,”and the system can store electronically the queries and response-optionsin a database of the back-end computer system, as described herein. Thecategories of investigative analysis can be broken down into subcategories, subjects or “chapters.” There can be, for example, dozens ofcategories/subjects, such as forty (40) or so for one transaction’sanalysis. A collection of one or more investigative scripts related to acategory (or content chapter) leads to a summarized score for thecollection of investigative scripts. The score may indicate, forexample, how well the documents resolve or addresses the various riskcontingencies covered by the document or the corresponding transaction.In various embodiments, the higher the score the better (e.g., lessrisky) the information features, although the scoring system could beset up so that lower scores are better. As shown below, theinvestigative scripts can be in the form of a flow chart or tree, sothat a particular response to one initial investigative script leads todifferent follow-up specialized inquiries than a different response tothe initial investigative script. That way, the analyst can efficientlycomplete the series of queries by not wasting time on irrelevantqueries/subject matter. Once the analyst completes the analysis, acomposite score for the available information can be computed based onthe scores for the individual collections of specialized inquiriesacross the various chapters. For example, in an embodiment where thesystems/methods are used to score the risk imbedded in a securitizationdocument, the score can indicate the risk embedded in the transaction’sfeatures. The specialized inquiries and associated scores can bedetermined or set by a content designer such as a subject matter expertin the field (e.g., a “designer”) as mentioned above.

The scoring that can be found at the end of an investigative analysis isusually a numeric score. In other versions of this invention, there canbe other benchmarking disclosures such as pass/fail, material vs.non-material, helpful vs. non-helpful, etc. For example, a numeric scorecan be converted to such classifications based on whether the numericscore is within the range for a particular classification.

FIG. 2 is a diagram of a computer system according to variousembodiments of the present invention. The computer system comprises aback-end host computer system 10 that receives data such as atransaction / financing related document 12 to be analyzed from a thirdparty 14. For the purposes of the description to follow, the buy/selltransaction / financing related documents 12 are assumed to be asecuritization document, and in particular an ABS, and the investigativescripts are designed to identify how the securitization document treatsthe various risk contingencies associated with that particular ABS. Itshould be noted, as explained further below, that the present inventionis not limited to being used with securitization documents, and it couldbe used with other types of buy/sell transaction / financing relateddocuments. Also, the transaction / financing related document 12 ispreferably in a human-readable, text-based format, such as pdf, XML or aword processing format (e.g., Word). If the document is notword-searchable at the time of uploaded, it can be made word-searchableby the optical character recognition tool described below.

The host computer system 10 may be implemented with one or a network ofco-located or distributed servers or other types of computer devices,such as mainframes, for example. The third party 14 may be, or may beassociated with, the issuer, the service provider and/or the lender forthe securitization, for example. The third party 14 may transfer thedata item such as a document 12 to the host system 10 in an electronicversion, such as pdf, via email, a file transfer system, or by any othersuitable means for transferring and storing copies of electronicdocuments. The host system 10 stores the document in a document database16. Also, the document 12 may be stored in physical or digital form in adata room (e.g., a virtual data room in the case of a digital document)or on a distributed ledger.

When an analyst wishes, or is tasked with, reviewing and benchmarkingvarious risk elements found in one or more transaction / financingrelated data elements such as documents 12, a document server 20 of thehost system 10 can display the document 12 on that party’s device 18,such as in a browser on that party’s device 18, in response to a requestfrom the party’s device 18 for the document 12. That party’s device 18and the host computer system 10 may be in communication via a datanetwork, such as the Internet, a LAN, a WAN, etc.

In various embodiments, the analysis script related to analyzing thebasket of allowable data sources, including documents, can be providedby a web- or HTML-based application provided by the invention’s scoringsystem 22 and a web server 24. The risk benchmarking system 22 may storeeach relevant specialized inquiry, including the pre-determined possibleresponses for each inquiry, as well as the combined specialized inquirypaths (as described further below) and compute the final benchmarkidentifier such as a numeric score for the combined risk-assessmentitems, along with individual subject scores for the special matter foreach applicable subject, based on the analyst’s responses to theinvestigative scripts. The analyst may access a pre existing library ofinvestigative scripts; use their own library of investigative scriptsvia a browser on the user’s computer device 18; or a combination ofsuch. A web server 24 of the host computer system 10 may serve web pagesto the analyst’s computer device 18 that contains the user interface forthe investigative scripts, and the web pages are rendered by the browserof the user computer device 18. Also, multiple different analysts canassess the system to score different subjects/categories of the sametransaction / financing related document simultaneously. For example,one user can use the system to score the transaction / financing relateddocument for the 1st subject/chapter, a second user can simultaneouslyuse the system to score the same transaction / financing relateddocument for the 2nd subject/chapter, and so on. To that end, the hostcomputer system 10 (e.g., the web server 24) may support multiplesimultaneous user sessions applicable to one matter or many differenttransactional / financial matters at the same time.

The host computer system 10 may also be programmed to exercise versioncontrol so that two different analyst cannot edit the inquiries andspecialized-response options that the investigative scripts for the samesubject/chapter at the same time. Additionally, there could be more thanone analysis content library.

Also as shown in FIG. 2 , the system may further includedevices/computers 19 for administrators, subject matter subject-mattercontent designers, application engineers, a supervisor(s) for theanalysts, etc. that are in communication with the host system 10 via anetwork connection. From the administrative computers 19, anadministrator, subject-matter content designer, application engineer,supervisor, etc. can make changes, updates, modifications, etc., to thesystem as described further herein. For example, a designer can revisethe queries and/or responses; an application engineer can modify thescoring algorithms; a supervisor can edits a final report and/or assigntasks to analysts, etc.

FIGS. 3, 4, 6 and 7 are screen shots showing examples of the userinterface provided by the scoring system. The screens shots may bedisplayed as web or html pages, for example, served by the web server 24to the user device 18. FIG. 3 shows a scoring summary page. Referring tothe key in the upper left in this representative example, the identifierfor the financing document being reviewed in this example is given bythe anonymized or non-anonymized “Transaction” number. The “Shelf” fieldmay indicate a party related to the transaction that provided orauthored the document. In the case of securitization documents, thatparty often is a financial institution such as a lender. The “CollateralType” field is particular for securitizations and shows the collateralin the securitization, in this example, residential mortgages. Thereport date field shows the last date that the report was edited.

The user can print out certain screen shots from the system and usethose print outs to communicate with her /his associates if they preferprinted versions. In various embodiments, the computer system may employsoftware code written in Python to convert HTML webpages to PDF formatfor printing. The Python can convert the HTML content for a web page toPDF using, for example, a SelectPdf HTML To PDF REST API through a POSTrequest, with the parameters being JSON-encoded. The resulting contentcan be saved into a file on a disk of the computer system for printing.

As mentioned above, the investigative scripts could be grouped intoprimary and secondary analysis categories or “chapters”. The exampleillustrated in FIG. 3 shows six possible chapters relevant to aresidential mortgage securitization-ESG (environmental, social andgovernance) risk, sponsor risk, collateral outliers, trustee risk,representation and warranty testing, governance and supervision. Inpractice, there could be dozens of different analysis categories orchapters. The example illustrated in FIG. 3 shows that there can be anumerical score for the chosen inquiry path associated with eachchapter, and the bar graph shown in FIG. 3 illustrates the score foreach chapter. The scores are derived from the responses to theinvestigative scripts in each chapter. How each response affects thefinal score can be set by the designer. Each of the bars in the bargraph may include a hyperlink, such that when the analyst clicks on abar for a particular subject or “chapter,” the analyst’s browser opensthe investigative script page for that subject/chapter so that theanalyst can commence the investigative research.

The scores for nodes and chapters are assigned upfront by the contentdesigner, stored in a database of the back-end computer system, and thenrefreshed manually and or automatically as evidence is found to justifya change in the score. Given the focus on risk, the scores tend to focuson downside risk exposures which in turn tends to bias to a lower scorefeatures. For instance, a party who provides their service to afiduciary standard of care could get a score of 95 (out of 100) yet anoperator that is a poorly capitalized could get a score of 40 (out of100) even though they are both providing a somewhat similar service.

An example investigative script page is shown in FIG. 4 . In theillustrated example, the investigative script page has a key 60 in theupper left that shows the subject of the investigation or chapter (inthis case, governance) for the particular product type (in this case, USRMBS or U.S. residential mortgage backed securities). The key 60 mayalso show the range of possible final score options for the subjectmatter of the particular category/chapter for the document (in thiscase, a final subject score range of 14 to 90 wherein, in this example,a higher score is better (less risk)). As shown in the example of FIG. 4, the investigative scripts are shown in a specialized inquiry view withnumerous nodes. The key 60 may show the total number of nodes in thespecialized inquiry for the chapter. The key may also show the totalnumber of nodes in the specialized inquiry for this category/chapter tobe reviewed, and the total number of nodes that result in scores. Thenodes to be edited are shown in white and the end a node is shaded grey.For each investigative and scoring path the analyst’s UI provides theinvestigative scripts that the user (referred to below sometimes as a“specialized investigator” or “analyst”) considers as she/he selects aspecialized-response option or provides a free form answer. In theillustrated example, the user (e.g., specialized investigator) worksfrom left (the beginning) in the field of possible nodes to the right(the end). In the instance that there is a reason to record whereconfirmatory or conflicting information related to an inquiry is found,the system, including the user interface, provides an infrastructure todo such. The user clicks on the left-most node 62, whereupon theinvestigative script is displayed in full for the user and the user isprovided a space or field in which to input, in text, for example, thesource or location where the responsive information was found. The firstspecialized inquiry in the illustrated example is whether there is aseparate entity that has no ongoing operational responsibilities butacts as a specialized agent. If the user determines thespecialized-response option is yes, the user enters a locationidentifier for where evidence supporting that determination was found(supporting evidence). The location identifier for the supportingevidence could be a verbal discussion, research paper, transactiondocument, a section within in a document or sub-section title, a pagenumber(s), and/or a section number(s) (e.g., Section V.A.3.b.ii(b)(2),etc.). The interface may further allow the user to provide a shortdescription of why the supporting evidence at the location identifierconfirms or conflicts with the chosen specialized-response. Thedescription and supporting evidence citations may help with analysisquality control and/or help the search functionality for futureanalysis. The scoring system 22 may store in the query/response databasethe specialized-response option entered by the analyst, the locationidentifier for the supporting evidence, and the description in a datafile that associates the response to each specialized inquiry/nodeidentifier for the transaction / financing related document for theshelf (e.g., bank). That way, as explained further below, the responsesrelated to the same specialized inquiry for different, but similar,transactions for the same and/or different banks could be queried,compared and analyzed for this and future transaction / financingreviews.

An edit node, such as node 62, includes a number of connected nodes tothe right, with the number of connected nodes indicating the number ofpossible response options to the specialized inquiries. Some specializedinquiries/nodes, such as node 62, can have two possible responses, suchas yes/no responses. Other nodes can be set up to have multiple possiblespecialized-response options, such as nodes 65 and 67A, for example.

Also, in various embodiments, the nodes may allow the user to input aconfidence score related to the evidence found for thespecialized-response option. The node may, for example, allow the userto input a numeric confidence score in a range, such as one to five, oneto ten, or one to one hundred, for example, with the higher the numberindicating a higher confidence on the part of the user that the user’sspecialized-response option for the analysis node specialized inquiry iscorrect. Low confidence scores can be used to prompt a second review ofthe data item such as a document by a more experienced or knowledgeableinvestigative analyst and generate a specialized notation in the outputreport. The confidence score can act as a filter when giving feedback tothe content designer who may use that information to update the systemfor future applications. A low confidence score may suggest amodernization to the system code may not be useful.

As shown in the example of FIG. 3 , there are two nodes leaving node62--nodes 63 and 64. Node 63 corresponds to an affirmativespecialized-response option at node 62 and node 64 corresponds to anegative specialized-response option at node 62. Thus, if thespecialized-response option to the first specialized inquiry at node 62is yes, the user then proceeds rightward to the next node 63 and repeatsthe process for the investigative script at node 63. Preferably, thescoring system software 20 does not allow the user to move to the nextnode until a specialized-response option is provided to the precedingnode. That ensures an orderly, linear and complete progression through aparticular investigative sequence. If the specialized-response option tothe investigative script at node 62 is no, the user proceeds to “end” or“terminating” node 64, which in this example terminates the specializedinquiry for this category/chapter with the score indicated by the endnode 64 (in this case, a score of 14). Sometimes a yesspecialized-response option can terminate the specialized inquiry, suchas shown at end nodes 66A-B. Such terminating yes nodes can be used toprompt a score for the category/chapter. Preferably, as shown in theexample of FIG. 4 , each path through the specialized inquiry leads to aterminating node and as associated score. The complete specializedinquiry is not shown in FIG. 4 as indicated by the ellipses for nodes67B-C and 68A-D.

FIG. 11 is another example screen shot showing the node tree accordingto various embodiments. A subject matter expert designing the queriesfor a chapter can use the color-coded node scheme of FIG. 11 , where,for example, similar types of nodes have the same color and/or where thelevels (or columns) of the tree are color coded. That is, in the latervariation, nodes at the same level (or column) can be the same color.

As mentioned above, in the investigative interface tool, the specializedinvestigator inputs the place in the information source content (e.g.,location identifier) where the specialized-response option to theinvestigative script is found. The scoring system 22 (e.g., a databasethereof) can store the individual responses in the document indexdatabase 26 (see FIG. 2 ). Further, the document index database maystore then export a copy of the responses in a spreadsheet or tableform, for example, that shows the location for the responses to eachspecialized inquiry. The metadata for the specializedinquiries/responses can include the transaction / financing relateddocument itself, the category for the specialized inquiry, thespecialized inquiry identifier, the product sector or collateral type(e.g., residential mortgages), and the shelf (e.g., the bank thatcreated the document), such as shown in the example of FIG. 5 . Thus,the spreadsheet/table could be sorted by Doc ID, product, bank, chapter,question ID, etc. FIG. 5 shows an abridged table with three prior RMBSdeals by ABC bank (Doc IDs 0001, 0003 and 0004) and one prior RMBS dealby DEF bank (Doc ID 0002). The table further shows for each specializedinquiry (specialized inquiry ID) in each category/chapter, where thespecialized-response option was found (response column). In theillustrated example, the location is shown as a page number of thedocument. In other embodiments, the location identifier for the responsecould be a section/sub-section title or number in addition to or in lieuof the page number, or any other suitable location identifier for thedocument. This information can be invaluable to other systems which loadthis computer system’s data into their computer system.

Accordingly, in various embodiments, when the user reviews thespecialized-response option in the specialized inquiry, the node mightadditionally display the most likely places where the supporting ordilutive evidence can be found based on responses from prior informationsources. FIG. 6 shows an example display of an edit node for a yes/noinquiry that is displayed when the user clicks on the edit node. Asshown in the example of FIG. 6 , the edit node display shows the nodenumber and the subject/chapter/category. It also shows the specializedinquiry and provides pre-programmed response option radio buttons 70 orother web page input icons (e.g., drop down menu item selections) thatthe specialized investigator could use to input the specialized-responseoption. The inquiry and corresponding specialized-response options canbe specified by a subject-matter content designer using the applicationcomputer device 19 (see FIG. 1 ). When, in performing a review of thesubject information and after the subject-matter content designer hasspecified the inquiries and corresponding specialized-response optionoptions, the user inputs one or more free form responses and or selectsone of the specialized-response options, which can be finalized bypressing the “Finalize & Submit” button 72. The scoring system 22 canstore the specialized-response option for the specialized inquiry/nodefor the document being reviewed by the specialized investigator.

The population of allowable items that can be searched is restrictedbased on the specialist’s data access privileges and the user’s outputreport credentials. Parties that can view only publically availableinformation should probably not have analysis or reports based onprivate or confidential information. Limits on the search authority arepart of this invention.

Traditional search rarely focuses on missing or omitted information yetmissing information can be a key item in assessing risk. This inventionhas analysis scripts for known drivers of risk. If there is nodiscussion of such in the documents, this omission could be a materialinfluence to scoring risk. Knowing what is omitted can be highlyvaluable in assessing and scoring risk.

As shown in the example of FIG. 6 , the response form for theinvestigation node can include a field 73 where the user enters thedata’s location identifier(s) in the source information data set, suchas a document, where the supporting evidence or authority for thespecialized-response option to the inquiry for the node is found. Theresponse form for the node may also include a free text block 74 wherethe user may enter relevant text explaining the usage and or value ofthe specified evidence for that specialized-response option. Theresponse form may also include a context field 71, which canshow/display to the specialized investigator some contextual orbackground information as determined by a subject-matter contentdesigner regarding the inquiry’s relevance and/or keywords relevant tothe specialized investigator’s analysis. This contextual and analyticalinstruction can make the specialist more sophisticated and aware of thenuances related to the matters associated with that specific inquiry.The keywords can be gleaned from explanations from reviews of prior,similar transaction / financing analysis. That is, the specializedinvestigators’ explanations are indexed to the specialized inquiries andstored in a database of the host computer system. A search featureapplied to that investigative node in the future can then search theexplanations for a particular specialized inquiry to determine the mostrelevant concepts related to specialized-response optioning the inquiry.A subject-matter content designer can use relevant concepts from thesearch in the context field 71 to assist the specialized investigator inspecialized-response options regarding that specialized inquiry.

Also as shown in the example of FIG. 6 , the display may include a field76 that shows the top location(s) where the supporting evidence islikely to be found based on queries of the information index database 26for: the same specialized inquiry subjects such as the same type of thecollateral; for the same bank etc. Secondary locations (in terms ofpriority) would be for the same supporting evidence, for the same typeof collateral, but for different banks. For each investigative node,more recent searches results can be weighted higher to speed up the userexperience in the future. For example, the top suggestion on where tofind responsive information could be a page range that encompasses thelast 5 or 10 responses to the specialized inquiry for the same type ofdocument (e.g., type of collateral) for the same bank. It could also oralternatively show the precise document location identifier whereevidence related to the specialized-response option was found for thelast 5 or 10 (or any number) of prior documents for same collateral typefor the same (or different) banks. The descriptions that the specializedanalyst inputs when reviewing prior transaction / financing relateddocuments can also be used by the investigative node’s search algorithmto identify the relevant transaction / financing related information todisplay in the field 76. Also, some or all of the displayed locationscould have a hyperlink that when clicked by the user causes the hostcomputer system 10 to retrieve and display (such as in a pop-up windowor in another web browser tab or window) the corresponding section ofthe other transaction / financing related document so that the user cancompare the documents to see if their specialized-response optionrelated to that inquiry in the same way or not.

Also, with sufficient data, transaction / financing related informationfrom different banks for the same type of collateral may look similar.That is, one bank’s RMBS documents could look similar to another bank’sRMBS documents. For each type of document (e.g., collateral type), andfor each bank in the database, the scoring system may compute asimilarity score to each of the other banks, where the similarity scoresare based on the similarity of the documents from the two banks for eachtransaction type. The similarity could be based on thespecialized-response options and where the supporting evidence is found.Thus, when reviewing a document for ABC bank, the investigative nodescould also show the most likely locations where the supporting evidencemay be found in the documents for ABC bank’s most similar bank(s) forthe particular loan/deal type. Similarity and identical documentsbenefic from different analysis methodologies. This invention providesfor such.

When the specialized investigator enters her/his response into theinvestigative tool, and finalizes the response by hitting the button 72,the information source database 26 can be updated accordingly. It canrecord the location identifier(s) for the information from the field 73and any explanation provided in field 74. When the specializedinvestigator hits the next inquiry button 72, the specializedinvestigator can then be automatically taken to the next specializedinquiry corresponding to the investigative tool given the specializedinvestigator’s selected response option to the instant inquiry.

In some cases, the data source, such as a document, may not provide asufficient specialized-response option for an investigative script(s).In those instances, the user can indicate in the explanation field 74that no specialized-response option was provided or the user can inputan alternative custom response. This input can create a prompt for thesubject-matter content designer and/or supervisor to update theinvention’s scripts for future applications. Due to the cyclical natureof the transactional and financial matters, this ability to input customoptions effectively causes the invention to become more and moreinsightful for future analysis. Although transactions and financingdocuments lack standards, they often include common features that followhistorical trends. For instance, contract features related to workoutstrategies may be written in detail when prime and sub prime collateralis included in a financing but over time, the reference to workouts onprime collateral may fall away. In periods of recession, the contractprovisions related to prime might re appear in future financings.

FIG. 15 shows another example of a query node that the analyst completeswhile completing a survey. The example shown in FIG. 15 is node number15553 (see upper right) in the particular query node tree. The query isquery 46.03 in the menu of queries (see FIGS. 12 and 13 where anadministrator generates the queries in the node tree for a transactionor transaction type). In the example of FIG. 15 , there are threepossible responses to the query. The analyst can click the “SupportingBackground & Context” button or link to, for example, see the possibleplaces to look for the answer to the query as in field 76 of FIG. 6 .The example of FIG. 15 also includes a place where the analyst can markwhether the substance of the query is important (or believed to beimportant by the analyst) and, as such, deserves further attention. Whenan analyst clicks or otherwise activates this button, the host computersystem 10 can send a notice to an administrator or manager to analyzethe issue further. The example of FIG. 15 also includes the button atthe lower left where the analyst can proceed to the next query in thequery node tree. As explained herein, the particular query node that isnext can depend on the answer given by the analyst for the instantquery. The means through which the analyst may specific a response tothe query can include the radio buttons as shown in the examples ofFIGS. 6, 7 and 11 ; drop-down menus; text fields; checkboxes; and/or anyother suitable (e.g., HTML webpage) response entry technique.

FIG. 7 shows an example of another edit node display. In the exampleshown in FIG. 7 , the inquiry has five option possibilities(“specialized-response options”). The specialized investigator can enterthe appropriate response by clicking the radio button 70 (or otherinput) corresponding to the appropriate response option. As before, thespecialized investigator can also enter the location of the informationwhere the supporting information to that response is found in the field73. In some applications, an inquiry could have dozens or even hundredsof responsive location options. Again, a subject-matter content designercan determine the appropriate weighting of the location options to beprovided to the specialized investigator and the user interface can beadjusted in the display of those options to the user as such. Thesubject-matter content designer can determine the appropriate optionsfrom, for example, a library of specialized inquiries and correspondingresponsive information options that are built up over time as finalmatters are reviewed and their corresponding evidence locations andexplanations are recorded in the database.

Supporting evidence is often broken into two categories: (1)confirmatory evidence and (2) dilutive, collaring, conditional, cappingor flooring evidence. In fact, the response to one node’s inquiry couldinclude both affirmative and diluting information and that informationmay be found in many different sources. The system’s ability to fund,link and use those multiple findings for future content enhancements isa useful part of this system.

In some instances, the specialized investigator may need to resort toevidence outside of the data set of base documents related to thematter. Depending on the type of information being reviewed, theextrinsic evidence could include industry research, public research(e.g., online searches of databases), specialized research, public orprivate bond offering documents, bond operating documents, loandocuments, borrower documents, operator document, tenant documents, etc.When electronic copies of these information sources are stored in thedocument database 16, the specialized investigator can identify where inthe extrinsic evidence or other the supporting, conflicting or dilutiveevidence for this is found. Where the supporting evidence are not in thedocument database 16, the specialized investigator could download themfor storage in the document database or otherwise indicate where thedocuments can be found for verification and/or audit purposes. In asimilar manner, the extrinsic documents can be word-searchable, such asby OCR-ing them with the OCR component 30 (described further below), tofacilitate computerized searching of the extrinsic evidence documents.In some versions, the OCR tool and a real person can look for theinformation together or separately. The interface can allow thespecialist to note if the data appears to be damaged, incomplete,unreadable etc. This can facilitate final analysis and follow ondecisioning.

In various embodiments, the listed responsive evidence citations infield 76 can have corresponding benchmarking scores that indicate thelikelihood that the responsive evidence citation will provide the properevidence to support the response option. The evidence confidencecitations can be scored based on, for example: how many times the priorevidence citation was observed; for the same issuer/bank; for similartransactions; for documents that are highly similar in general; and withmore recent citations being weighted higher to facilitate future search.

Also, although not illustrated in the examples of FIGS. 6 and 7 , thespecialized investigator’s interface screens may include a feedbackbutton where the specialized investigator could provide feedback to thesubject-matter content designer and/or a business supervisor. Thefeedback loop process may include, for example: suggested revisions tothe inquiry; suggested changes to the possible specialized-responseoptions (including deletions, additional and/or rewordings); and/orinquiries to be added or subtracted. This computerized feedback loopcreates continuous learning for the system.

When the specialized investigator completes the full path of specializedinquiries for an investigative subject, the benchmarking or scoringsystem 22 can generate a screen shot, digital, paper or other report forthe subject/chapter. FIG. 8 shows an example of how a subject-mattercontent designer can set up the specialized inquiries in the scoringsystem 22 to generate the analysis output report. The example of FIG. 8indicates the node identification number and investigative subjectchapter at the top. The subject-matter content designer can write, editor update the menu of investigative scripts in field 79. There can bechecklists per node. The display also includes an indicator 80 thatprovides a description to the score range for the risks related to thespecified matter as addressed for this specific node (i.e., node 1097 inthis example) in this chapter/category. That is, in this example, alltermination nodes downstream from this node (e.g., to the right in thepath flow) could have a scoring value of between 40 and 100 in thisexample. Further, the example in FIG. 8 has four possiblespecialized-response options 70. Each of the possiblespecialized-response options can also show the score range correspondingto each specialized-response option. For example, if the specializedinvestigator selects Certificate Holders as the specialized-responseoption, when the specialized investigator proceeds to the next node (byclicking the button 72 in FIG. 7 ), all downstream termination nodesfrom the next node will have a value of between 75 and 100. If, however,the specialized investigator selects Tax Residual Holder as thespecialized-response option, all downstream termination nodes from thenext node will have a value of between 60 and 90, and so on for theother specialized-response option alternatives shown in the example ofFIG. 8 . The chart 82 in the example of FIG. 8 also shows the contentdesigner how the specialized-response option will affect the overallscore for the category for the risk being analyzed. For example, withreference to the example in FIG. 8 , if the specialized investigatorselects “Directing Certificate Holder” as the specialized-responseoption, the chart 82 shows that the final score range for thechapter/topic (which depends on the other specialized-response optionsalong the specialized inquiry branch) is between 40 and 65. Conversely,if the specialized investigator selects “Certificate Holders” as thespecialized-response option, the final score range is between 75 and 100(with higher numbers indicating less risk in this example).Collectively, as the specialized analyst progresses down the path ofinquiries, the final score benchmark options start to show a trend andbecomes refined tighter and tighter as the path is concluded. Thiseducation of resulting score results, during the analysis process, helpskeep the specialized analyst cognitively aware of the impact of her/hisselections to date. This education can assist in free form answers at alater date,

The chart 82 in FIG. 8 also has a “Long response option” for each shortresponse option. The Long response option corresponding to a shortresponse option selected by the specialized investigator when reviewingthe applicable evidence shows up in the chapter summary report asdescribed further below. The subject-matter content designer can editthe long response options in the chart so that they are appropriatelyworded for the resulting chapter summary report. In particular, the longresponse option preferably gives support and/or context for the shortresponse option.

The designer UI includes a library of common answer options. Whendesigning the content of a node, that library can be referenced and thecontent designer is provided tools to copy that content into the answerarea of any investigative node. As more subjects and matters areanalyzed that content library grows, the library allows the designer tobe faster when designing a new node and facilitates consistent contentexporting when a survey is completed. The designer’s includes an optionto make the copied over list of answer options static for that node ordynamic. A dynamic authority will cause the node’s chosen answer optionlist to get longer or shorter as the base library for that item ismodified in the future. For instance, the library could include the nameof all 50 states and the list is simply copied over if there is aninquiry related to state location.

The example of FIG. 8 also has a section 84 where the subject-mattercontent designer can indicate the importance of each responsive optionto the overall risk of that analysis “chapter” or the risk as a whole.FIG. 8 shows one section 84 for doing this, although the contentspecialized investigator preferably completes this section for eachpossible specialized-response option 70. The section 84 in the exampleof FIG. 8 allows the subject-matter content designer to display thesubject-matter content designer’s pre-determined view of the importance(e.g., a rating of 1 to 5, with 5 being most important) of eachspecialized-response option and whether the consequences of the selectedresponse option are positive or negative to the overall risk profile ofthe matter being analyzed. This information is used in generating thereport for the analysis “chapter” as described further below. In such ascenario, a first particular specialized-response option to a particularinquiry may be highly important while a second, differentspecialized-response option to the same specialized inquiry is not asimportant. In various embodiments, the display of FIG. 8 allows thesubject-matter content designer to specify the importance on aspecialized-response-option by another specialized-response optionbasis, with the long response option and importance rating for thatresponse option selected by the investigator at review time appearing inthe chapter summary report as explained further below.

FIGS. 12 and 13 show screen shots of the user interface that anadministrator, at the administrator device 19, can use to pre-storedquery node trees for various subjects to establish which topics (orchapters) are used (analyzed) for a particular transaction ortransaction type. FIG. 12 shows how the administrator can selectpre-established query node trees for various subjects within and acrossvarious applicable topics (e.g., legal, legal loan, financialcollateral, etc.). The user interface can show, as shown in FIG. 12 ,the version for each query node tree, the last time the query node treewas modified, the number of dynamic and static nodes in the query nodetree, etc. FIG. 13 shows an example user interface where theadministrator has completed the selection of investigative topics fromFIG. 12 . FIG. 14 shows a screen shot that an analyst (from analystdevice 18) may use to sort and start the surveys/jobs assigned to theanalyst. When clicking the “Start Survey” hyperlink for a particularjob, the webpages with the queries assigned to that job by theadministrator (see FIGS. 12 and 13 ) can be retrieved and conveyed tothe analyst (at the analyst client device 18) for completion of theapplicable surveys as described herein.

FIG. 9 shows an example report for an analysis chapter, which can begenerated by the scoring system 22 after the specialized investigatorcompletes the specialized inquiries for a chapter. The report compilesand displays the response options in the chart 82 that corresponds tothe response option selected and where the subject-matter contentdesigner specified her/his long specialized-response options. Also, thereport aggregates the long response options by significance/important,as being either positive, negative, or simply notable and then withineach of the positive, negative and notable categories, by importancerank, e.g., most important (e.g., rank 5 in FIG. 8 ) to less important(e.g., rank 1 in FIG. 8 ). As shown in the example of FIG. 9 , thereport can compile, as “Conclusions,” all of the response options in thechart 82 for positive features from rank 5 to 1, and all of the responseoptions in the chart 82 for negative features from rank 5 to 1. The samefor notable. As shown in FIG. 9 the report can also show the specializedinquiry node number for the node that gave rise to the correspondingconclusion, as well as the next node that resulted from the specializedinvestigator’s response to that specialized inquiry.

For reference, the report can also show the chapter/category (in thisexample, representation and warranty testing), the sector (US structuredproducts in this example), the specialized investigator’s name, thereport date, the transaction id, the issuance shelf (or bank), and/orthe collateral type (residential mortgage in this example), as well asother information that may be useful.

Referring back to FIG. 4 , the termination nodes shown there correspondto responses to simple yes/no inquiries. In various embodiments, abranch of the specialized inquiry could terminate with a response to aspecialized inquiry that is not a yes/no response, such as shown in theexample of FIG. 7 . For example, one or more of the specialized-responseoptions in the example of FIG. 7 could terminate a branch of thespecialized inquiry and have an associated score for the analysisbranch.

In various embodiments, the scoring system 22 may also comprise an OCRcomponent 30. The OCR component 30 may OCR a transaction / financingrelated document or other information including those listed previouslythat is not otherwise word-searchable such that the words of thedocument are word-searchable (for documents that are not alreadyword-searchable). That way, for a particular specialized inquiry, thescoring system 22 can do a word/phrase search for the relevant termsthat are responsive to the particular investigative scripts.

The documents and/or each page thereof stored in the database 16 may beits own file, such as a PDF, BMP, TIFF, JPEG, and PNG files, forexample. The OCR component 30 processes the files to recognize thecharacters and the words in the files so that the contents of the filescan be searched. As a first step, the OCR component loads the files tobe OCR-ed. Depending on the method in which the image files werecreated, there are a number of issues that may arise. More often thannot, an image file will be skewed or contain “noise” (a/k/a varyingbrightness or color). As a second step, the OCR component preprocessesthe image files to, for example, de-skew, remove any “noise”, andimprove the overall quality of the images. In various embodiments, thepreprocessing step can include the detection and removal of lines on theimages/pages, which tends to allow for better recognition quality whenconverting tables, underlined words, etc.

Next, the OCR component 30 analyzes the page/image being OCR-ed. In thisstep, the OCR component notes and processes the layout of the originalfile, including the detection of text positions, white space, and theprioritization of important text areas or sections. The aim of thesepre-processing steps is to convert the file to a binary file - that is,every pixel on the image is one of two colors (e.g., black or white).The white areas can be ignored, while the black areas are analyzed todetect the characters. Next, OCR component 30 detects (or singles-out)words and lines of text in the file as a beginning stage of actualcharacter recognition. Next, the OCR component 30 may detect and fix“broken” or “merged” characters. Depending on the quality of theoriginal file, there are often errors in which characters are broken orblurred together. The OCR component 30 may break down and resolve theseerrors in order to properly interpret the appropriate characters.Finally, once individual characters are identified, the OCR componentrecognizes the characters. The OCR component may use matrix matchingand/or feature extraction for this step. Matrix matching (or patternmatching) identifies the image-based files as the equivalent plain textcharacter when an image (a stored collection of bitmapped patterns oroutlines of characters) corresponds to one of these selected bitmapswithin a certain degree of likeness. Alternatively or additionally, theOCR component may use feature extraction, which searches a character onthe page for common elements, like open spaces, closed forms,lines-diagonals intersecting, etc. to recognize the character. Usingeither (or both) technique, the OCR component initially advancesnumerous hypotheses about what a character is. Based on these hypothesesthe OCR component analyzes different variants of breaking of lines intowords and words into characters. After processing huge number of suchprobabilistic hypotheses, the OCR component finally makes the decision.When a character is identified, the OCR component 30 can convert it toASCII code so that it can be used for further manipulations, such as theidentification of words from the recognized characters using adictionary.

Where the “documents” to be search include other types of media, such asaudio or video content, the computer system can use automatedtranscription software to convert the audio to text that can beprocesses and searched. The transcription software can use naturallanguage speech recognition, for example, to convert speech in an audiofile to text.

The scoring system 22 can learn the relevant terms for inquiries basedon the prior responses to the same inquiries for the same bank, or, lesspreferably, from a different bank. For example, referring to FIG. 5 ,the response option to the specialized inquiry # 001in the Governancechapter for ABC bank for a RMBS for the last such deal (Doc ID 00004)was a page 73. For a new deal (with a new transaction / financingrelated document) by ABC bank for RMBS, the scoring system can searchthe document for the new deal (e.g., Doc ID 00005) for terms and phrasesthat were found on p. 73 of the prior deal’s document (Doc ID 00004).The text or page numbers of the top scoring section(s) of the newdocument could be presented to the specialized investigator in the nodebox (e.g., see examples at FIGS. 6-7 ) for specialized inquiry # 001 sothat the specialized investigator has a smart, initial location to lookfor the specialized-response option. The node box could also include alink to the top scoring section(s) in the transaction / financingrelated document 12.

The content designer upfront and specialist user on going can input orhighlight helpful key word and search criteria into the invention. Thissearch list can be automatically enhanced as the user finds, highlightsand stores responsive sentences, paragraphs and sections that areresponsive. Effectively, the OCR search grows in sophistication andspeed as responsive contract test is highlighted and stored in theword/phrase database. The OCR finding that nothing was responsive tothat inquiry’s data needs is equally important to know sometimes.

In addition, in various embodiments, the scoring system 22 may comprisea document (or text passage) similarity comparison module 31 as shown inFIG. 2 . The document similarity comparison module 31 may compare theword-searchable subject found (e.g., a particular text passage of adocument) in a document 12 to prior word-searchable documents, stored inthe document database 16, from the same (or different) bank for the same(or similar) collateral type to identify text passages in the subjectdocument that are similar to the passages in the prior transaction /financing related documents where the specialized-response option to theinquiry appeared. For example, if the answer location for specializedinquiry # 001 in the Governance chapter for ABC bank related to a RMBSfor the last such transaction / financing related document (Doc ID00004) was a page 73, the document similarity comparison module 31 cancompare the text at p. 73 of the last transaction / financing relateddocument to the current (subject) transaction / financing relateddocument to find the most similar (or the N most similar, where N > 1)text passages in the current transaction / financing related documentand present them to the specialized investigators in the display for thenode for the specialized inquiry as initial places to look in thecurrent document 12 for the specialized-response option. This could alsobe done for the last M transaction / financing related documents (whereM ≥ 1). The document similarity comparison module 31 may use anysuitable technique for comparing passages of the documents using NaturalLanguage Processing techniques, such as Jaccard or cosine similarityscores.

In some embodiments, the scoring system 22 may automatically providespecialized-response options based on the similarity between the newdocument and the immediately prior transaction / financing relateddocuments for the same bank for the same product (or a number of priortransaction / financing related documents). For example, if the newdocument being scored contains a passage that is sufficiently similar(e.g., a similarity score above some threshold, as determined by thedocument similarity comparison module 31) to the passage of the priordocument that contained the specialized-response option to thespecialized inquiry, then the scoring system 22 can specialized-responseoption the specialized inquiry in the same manner as the prior document.The benchmarking and scoring system 22 can also provide a confidencescore that is related to the similarity score (e.g., the higher thesimilarity score, the higher the confidence). Also, instead of one datasource being used, the automatic specialized-response option andconfidence score could be based on more than one document, e.g., thesimilarity to the last N documents for the same collateral type for thesame bank, etc.

When a similarity score between the document being analyzed and a priordocument (or the similarity scores between the document being analyzedseveral prior documents) is very high, e.g., above a threshold scorelevel, and the analyst responses for queries focusing on the relevantpassages are consistently uniform, the scoring system can select theappropriate answer (the prior consistently uniform answer) andcorrespondingly move to the next relevant query in the node tree. Thatfunctionality accelerates the review by the analyst. The systemautomatically selects the response to the query, thereby absolving theanalyst from having to spend time on the query.

In a related manner, particularly for queries that have many possibleresponses (as opposed to merely yes/no responses), the back-end systemcould reorder the order in which the possible responses appear to theanalyst, so that the most likely responses appear at the top of the userinterface. For example, when a few or a handful of responses predominantfor a query, based on the analysis by the comparison module 31, thosepredominant responses can be shown at the top of the analyst’s listing.In other words, the comparison module 31 could compute a likelihood ofresponses to a query, based on a comparison of the relevant sections ofthe document being reviewed to prior, similar document(s) that werescored, and the corresponding responses from the prior document(s), andthen display the responses for the analyst in descending order oflikelihood. This is another efficient aspect of the user interface; itcan speed the analyst’s review and response to a query.

In various embodiments, the content designer could specify a glossary ofkey terms and, in turn, the key terms that are relevant to a particularquery. In performing its qualitative comparison of text passages, thecomparison module 31 can weight the specified glossary terms greaterthan non-glossary terms. In addition, in various embodiments, thecontent designer could specify “dilutive” terms for specific queries,such that they are essentially “linked” to the “affirmative” key wordsfor the query specified by the content designer as described above. Thecomparison module 31 can apply a penalty when a dilutive term is foundin the document so that the dilutive (or counter-) effect of the founddilutive term(s) is(are) factored into the scoring.

After the specialized investigator completes the investigative scriptsfor each of the chapters, the scoring system computes a composite scorefor the document. The composite score can be a weighted average of theindividual chapter scores, with the more important chapters (e.g., interms of risk) being weighted more highly. Moreover, the weights canvary with time. For example, for a new securitization, the rep &warranty provisions may be more important upfront than the terminationprovisions for the SPV. However, years into the securitization, when thesecuritization is close to expiration for example, the SPV terminationprovisions may be more important and can be weighted higher for purposesof computing the composite score. As such, a specialized investigatormay review the documentation for a transaction at various times duringthe life of the transaction (which may last 10-20 years, for example).The investigative inquiries that are immutably based on the transaction/ financing related document, i.e., static information, do not change,so the specialized investigator does not need to redo those specializedinquiries. Alternatively, the specialized-response options to somespecialized inquiries, e.g., specialized inquiries about collateral, maychange over time; that is, the collateral items may be dynamic (e.g., isthe collateral continuing to be in working order, have all taxes beenpaid on it, etc.). In subsequent reviews, the specialized investigatorcan update the responses to those specialized inquiries. Moreover, asmentioned previously, since the risks may change over time, theweightings for the category/chapter risks may change over time, so thatthe composite score for a transaction / financing could change overtime.

Preferably, over time, the specialized inquiries can be modified, newspecialized inquiries can be added, old specialized inquiries can beremoved, and/or scores for a path can change as more information becomesavailable. For example, if there is a change in applicable law thatmakes additional specialized inquiries relevant or makes old specializedinquiry obsolete, a programmer/subject-matter content designer for thescoring system can edit the specialized inquiries or specialized inquirypaths to changes the specialized inquiries and/or the flow paths througha specialized inquiry. The content designer could also change theresulting scores for a path to reflect updated perspectives on the risksfor each path. The changing weighing of each subject chapter implicitlycreates a process of connecting independent neural nodes, which in turncreates digitized intuition.

In that connection, if a new product comes along where there are noprior information sources directly on point, the investigative scriptsfor the new product could be created by editing the existing inventoryof specialized inquiries and path scores for the most similar existingtransaction/product. For example, if a new product requires x newspecialized inquiries at certain points in the survey for certainsubject chapters, and there are specialized inquiries from the oldversion that are irrelevant to the new product, the content designercould create the investigative scripts for the new product by editingthe investigative scripts from the old base inquiry (and path scores ifnecessary) to add or delete specialized inquiries as appropriate toaccommodate the new product issues. In addition to the node managementwithin file 1, the chapters of specialized inquiries could be added ordelete too in such a manner. Moreover, the weights for the compositescores could be changed for the new product. The use of an old inquiryto act as a base inquiry document is that it saves time and money.

FIG. 10 is a flow chart illustrating a process of scoring a transaction/ financing related matter according to various embodiments of thepresent invention. At step 100, the subject-matter content designer(s)design the specialized inquiries, the specialized inquiry paths, and thepath scores for each chapter. The subject-matter content designer(s) maydesign the specialized inquiries as described above in connection withFIG. 8 . In this example, assume that there are N chapters. Thesubject-matter content designer(s) also determine the weights for eachof the chapter scores in determining the composite score for atransaction / financing related document.

At step 101, a specialized investigator downloads, from the documentdatabase 16 via and document server 20, one or more transaction /financing related documents 12 to be analyzed. A transaction / financingrelated document 12 may be for a new transaction, in which case has notbeen previously reviewed; or it could be a transaction / financingrelated document for a transaction that is already underway andpreviously reviewed (e.g., the securities have already been issued). Assuch, at step 102 the scoring system 22 determines whether the documentis for a new transaction or not. The scoring system 22 can make thisdetermination based on whether there are prior score data elements forthe transaction / financing related document indicative of prior scoringby the same or different specialized investigator. In either case, thechapter counter n is initially set to zero at steps 103 a-b, and thenincremented by 1 at steps 104 a-b.

At step 105, the investigator’s responses for the specialized inquiriesfor Chapter n are received and stored; at step 106 the scoring system 22determines the score for the transaction / financing related documentfor Chapter n; and at step 107 the scoring system generates the reportfor Chapter n (e.g., see FIG. 9 ). As described above, the scoringsystem 22 may provide the specialized investigator(s) prompts for wherethe specialized-response options to the investigative scripts may belocated based on prior transaction / financing related documentsinvolving the same (or similar) subject matter (e.g., RMBS) and/or thesame (or similar) bank. If the specialized investigator has completedall of the chapters (e.g., n=N at step 108), the scoring system 22 canthen compute the composite score for the transaction / financing relateddocument at step 109 by averaging the scores for each chapter accordingto the appropriate weighting scheme for the subject matter (e.g.,product) and the timing of the transaction (e.g., new, post-issuance,near maturity, etc.). If at step 108 n does not equal N, that is thespecialized investigator has not completed the last subject chapter, theprocess returns to step 104a where the chapter counter is incremented by1 and steps 105 to 107 are repeated for the next subject chapter. Thisprocess is repeated until all of the chapters are scored.

Returning to step 102, of the transaction / financing related documentbeing reviewed was previously scored/reviewed, the process is similar tothat described above, except that, as also described above, thespecialized investigator does not need to score the node or chapter thatare static (i.e., non-dynamic). For chapters related to static dataitems (which is different that the static chose-from library items), thescoring system 22 can use the chapter scores from the prior review.Thus, at step 110, the scoring system 22 determines whether Chapter n isdynamic or static. The subject-matter content designer can specify atstep 100 whether particular chapters or nodes within a chapter arestatic or dynamic and at step 110 the scoring system checks the settingspecified by the subject-matter content designer at step 100 for Chaptern. If the node or chapter is static, at step 111 the scoring system 22retrieves the chapter score and report for Chapter n from the mostrecent prior analysis.

One the other hand, if the chapter is dynamic, at step 112 thespecialized investigator completes the analysis inquires for the chapterand the application computes the chapter score in a manner similar tostep 105; at step 113 the scoring system determines the score for thechapter in a manner similar to step 106; and at step 114 the scoringsystem generates the chapter report in a manner similar to step 107. Ifall of the chapters are scored, that is, if n=N at step 115, thecomposite score for the analysis is computed at step 109. If not all ofthe chapters are completed, i.e., if n does not equal N at step 115, thechapter counter n is incremented by one at step 104b and the process isrepeated until all of the chapters are scored.

The host computer system 10 preferably stores the assigned risk scoresfor a transaction / financing related item that it is scored. Forexample, if a transaction / financing related document is scored atmultiple different times (i.e., to assess dynamic risks), the scores foreach review can be stored, with a time stamp indicating the time of thereview. That way, the change in the scores over time can be assessed asthe transaction / financing seasons.

The individual transaction / financing related subject scores, datascores and final reports, as well as the compendium of knowledge thatthe host system builds up over time from storing thespecialized-response options, etc. can be of tremendous value. First,the individual scores for a particular transaction / financing relateddocument provide insight into the risk associated with the particulartransaction. Also, the risks between different transaction / financingrelated documents can be compared / benchmarked to see where riskasymmetries occur. This information can help identify, project andmitigate action that can be taken to reduce the risk or increase thereward.

Second, the compendium of knowledge that the host system builds up overtime can help the specialized investigator review the data, such asdescribed above, such as by providing suggestions to where thespecialized-response options to particular investigative scripts can befound. Also, as should be evident, such a set up reduces the time forthe specialized investigator to review the complex documents and resultsin more accurate analysis.

Ongoing comments and explanations from the specialized investigators canalso be used to inform the system designer to improve improved theinvestigative scripts; the paths of the specialized inquiries, and/orthe associated assigned risk scores to better reflect differentsituations.

In various embodiments, network security measures can be used to controlaccess to the host control system 10. For example, only authorized usermay be permitted to edit the content of a node, modify a path of nodes,upload documents to the system and/or access the documents (e.g., thetransaction / financing related documents). Also, only certainauthorized users may access the final reports for a particulartransaction / financing related document to maintain confidentialityand/or propriety. The system engineer can manage all of thoseprivileges.

The invention allows the system owner to use the functionality of thesystem. Additionally, the system can be made available to third partieswhere the third parties can be given access to the system on a licenseor SaaS basis. In that instance, the third party could create and hosttheir own database of investigative scripts within the invention’sdatabase and or use the host system’s library of pre-existinginvestigative threads or combine both. The third party would be givensupervisory and designer interface features for their local environmentand links to connect such to the system’s content designer and systemengineer.

FIGS. 16-19 show examples of screen shots that can be displayed to ananalyst and/or administrator following completion of the survey for atransaction. FIGS. 16 and 17 are part of a single screen that the usermight need to scroll through. The middle part of FIG. 16 shows a“speedometer” gauge that shows the composite risk score for thetransaction as well as how it compares to peer transactions both interms of the point difference and the percentage difference. The bottomof FIG. 16 and continuing to FIG. 17 shows the score for eachtopic/chapter in the survey. The scores are shown with both numbers andbar graphs. The user interface can also include links where the user(e.g., analyst or administrator) can click to get detailed reports(e.g., the survey query responses) for the particular topics in thesurvey. The user interface can also include a count of the number of“notable findings” in each topic (e.g., see field 84 of FIG. 8 ). Tothat end, FIG. 18 is a screen shot that tabulates the “notable findings”and includes the importance rank assigned to each notable finding by theanalyst. FIG. 19 show the composite score for the transaction (in thisexample, a score of “66”). This page, for example, can be exported to beincluded with the paperwork for the transaction so that the parties areaware of the objectively-scored risk.

The benchmarking and scoring system 22 may be implemented with one or anumber of network computers, such as servers, mainframes, PCs, PDAs etc.Each computer of the scoring system 22 may comprise one or moreprocessors (e.g., CPUs or GPUs), primary data storage or memory (i.e.,memory that is directly accessible to the CPUs/GPUs, such as RAM, ROM,registers, cache memory), secondary data storage (i.e., data storagethat is not directly accessible by the CPUs/GPUs, such as HDDs, flash,SSDs, etc.), near line and/or off-line storage. The scoring system 22may be programmed to perform the functions described herein withsoftware that is stored in the primary, secondary, near line and/oroff-line data storage and executed by the processor(s) of the scoringsystem 22. For example, software for the OCR component 30 and thedocument similarity comparison module 31 may be stored in the datastorage and executed by the processor(s). The computer software may beimplemented using any suitable computer programming language such as.NET, C, C++, JavaScript, Python, Ruby, Lua, and Perl, and usingconventional, functional, or object-oriented techniques. Programminglanguages for computer software and other computer-implementedinstructions may be translated into machine language by a compiler or anassembler before execution and/or may be translated directly at run timeby an interpreter.

In one general aspect, therefore, the present invention is directed tocomputer systems and computer-implemented methods for providing animproved, efficient graphical user interface (GUI) to an analyst taskedwith reviewing one or more transactional documents of a transaction forrisk. In various implementations, the computer system comprises (i) ananalyst computer device that comprises a browser program; and (ii) aback-end computer system that is in communication with the analystcomputer device. The back-end computer system comprises: (a) atransaction document database that stores the one or more transactionaldocuments of the transaction in word-searchable form; (b) a querydatabase that stores pre-determined queries for the analyst toinvestigate in the one or more transactional documents for thetransaction, where the for at least some of the pre-determined queries,the query database also stores corresponding suggestions in the one ormore transaction documents for the analyst to review to respond to thequery, and where the suggestions are based on prior reviews oftransactional documents for similar type transactions; and (c) aweb-server for serving interactive webpages to the analyst computerdevice that are displayed by the browser of the analyst computer device,where the interactive webpages comprise an interactive query node treewebpage that display an interactive query node tree.

In various implementations, each query node in the interactive querynode tree corresponds to a separate query designed to assess risk forthe transaction and wherein each query node comprise a hyperlink. Also,upon the analyst activating the hyperlink for a first query node in theinteractive query node tree webpage, a corresponding query for firstquery node is displayed in a first query webpage.

The first query webpage can comprises means for the analyst to enter aresponse to the first query; an evidence field for the analyst to cite acitation in the one or more transactional documents that supports theresponse to the first query; a suggestion field suggesting one or moreplaces in the one or more transactional documents for the analyst reviewto determine the response to the first query; and a next query selectionbutton that, when activated by the analyst, cause a second query webpageto be displayed to the analyst, where the query for the second querywebpage depends on the response by the analyst to the first query.

The second query webpage can similarly comprise: means for the analystto enter a response to the second query; the evidence field for theanalyst to cite a citation in the one or more transactional documentsthat supports the response to the second query; the suggestion fieldsuggesting one or more places in the one or more transactional documentsfor the analyst review to determine the response to the second query;and the next query selection button that, when activated by the analyst,causes a third query webpage to be displayed to the analyst. The meansfor the analyst to enter the responses for the first and second queries(and any other queries) can comprise radio buttons, drop down menus,free text fields, HTML checkboxes, HTML select fields, etc. The back-endcomputer system is further configured to compute and display (or causedto be display on a computer device (e.g., analyst, administrator,supervisor computer device in communication with the back-end computersystem)) an overall risk score for the transaction based on theanalyst’s responses to queries in the query node tree.

In various implementations, the computer system further comprises anadministrator computer device that is in communication with the back-endcomputer system, where the administrator computer device comprises abrowser for displaying administrator webpages provided by the web serverof the back-end computer system, where the administrator webpagescomprise user interfaces through which an administrator specifies thequeries for each query node of the query node tree and an associatedquery score for possible responses for each query node, and where theback-end computer system is configured to compute the overall risk scorebased on the associated query scores for the responses provided by theanalyst to the queries.

In other various implementations, the query tree node specifies aprogression of query nodes. Also, the first query webpage may include anadditional field through which the analyst is permitted to flag that anissue related to the first query is important to the risk assessment.The additional field may further permit the analyst to enter animportance score for the first query. Still further, the back-endcomputer system may be configured to generate a final risk assessmentfor the transaction, where the final risk assessment comprises theoverall risk score for the transaction and a list of issues flagged bythe analyst as important to the risk assessment.

In various implementations, the back-end computer system is configuredto store the analyst’s citations in the evidence fields and use theanalyst’s citations in the evidence fields as suggestions in asubsequent risk assessment analysis for a second, similar-typetransaction. Additionally, the back-end computer system may be furtherconfigured to compare a passage of the one or more transaction documentsto a transaction document from a different, similar transaction todetermine the one or more places in the one or more transactionaldocuments specified in the evidence field for the analyst to review todetermine the response to the first query. In addition, the back-endcomputer system comprises an OCR module to OCR the one or moretransaction documents to make the one or more transaction documents wordsearchable.

The user interface provided by the present invention provides manyadvantages over existing techniques for reviewing transaction documentsfor risk, including in terms of efficiency. By having the progression ofqueries determine a priori according to the query node tree, theanalysts can efficiently progress from relevant query to relevant querywithout getting bogged down in irrelevant queries in the often verycomplex transaction documents that are often written in a style that isdifficult for a human to review and comprehend. Also, by including thesuggested citations for where the analyst should look to find a responseto the query, the user interface accelerates the review process. Thisfeature greatly accelerates the time to review the complex transactiondocuments. Also, by storing the analyst’s responses and evidencecitations, the system administrators can improve the queries and thequery flow (e.g., the progression of the node tree) as part of afeedback loop to make the analysis qualitatively better and moreefficient for the analysts. Also, the user interface is rooted intechnology. For example, it can utilize word-searchable electronicdocuments; it can include an OCR module for converting non-wordsearchable documents to a word-searchable form; it can utilizeinteractive web pages to present the queries, capture the analysts’responses in an efficient manner, and implement the query flow. Also,where the comparison module 31 computes that a responsive passage of thetransaction documents being reviewed are very similar to prior, similar,transaction documents, and the responses to a query in those priortransactions were consistently uniform, the system can deduce that theuniform response from the prior analyses is the proper response to thequery, and automatically enter the response and move to the next queryin node tree, thereby accelerate the review time of the analyst incompleting the query node tree. Also, the possible responses can besorted by likelihood, so that the most likely responses are listed first(higher), which also facilitates the analyst’s investigation. These andother benefits and technology features realizable through the presentinvention are apparent from the description herein.

The examples presented herein are intended to illustrate potential andspecific implementations of the present invention. It can be appreciatedthat the examples are intended primarily for purposes of illustration ofthe invention for those skilled in the art. No particular aspect oraspects of the examples are necessarily intended to limit the scope ofthe present invention. Further, it is to be understood that the figuresand descriptions of the present invention have been simplified toillustrate elements that are relevant for a clear understanding of thepresent invention, while eliminating, for purposes of clarity, otherelements. While various embodiments have been described herein, itshould be apparent that various modifications, alterations, andadaptations to those embodiments might occur to persons skilled in theart with attainment of at least some of the advantages. For example,additional applications for the above-described system could beapplicable to but not be limited to the medical, judicial or behavioralscience fields. The disclosed embodiments are therefore intended toinclude all such modifications, alterations, and adaptations withoutdeparting from the scope of the embodiments as set forth herein.

What is claimed is:
 1. A computer system for monitoring one or moretransactional documents of a first transaction for changing riskprofiles relative to prior transactions of a similar type, the computersystem comprising: an analyst computer device for an analyst that istasked with analyzing the transactional documents of the firsttransaction for risk to parties to the first transaction, wherein theanalyst computer device comprises is configured to display a graphicaluser interface (GUI) for the analyst to analyze the transactiondocuments; and a back-end computer system that is in communication withthe analyst computer device, wherein the back-end computer systemcomprises: a transaction document database that stores the one or moretransactional documents of the first transaction in word-searchableform, wherein the back-end computer system comprises an OCR componentfor converting transactional documents that are not word-searchable toword-searchable form, wherein the OCR component identifies individualcharacters in the transactional documents and identifies equivalentplain text characters for the identified individual characters usingmatrix matching and/or feature extraction; a query database that storespre-determined queries for the analyst to investigate in the one or moretransactional documents for the first transaction, wherein for at leastsome of the pre-determined queries, the query database also storescorresponding suggestions in the one or more transactional documents forthe analyst to review to respond to the query, wherein the suggestionsare based on prior reviews of transactional documents for priortransactions that are similar to the first transaction; and a back-endserver for serving interactive query node tree displays to the analystcomputer device that are displayed by the analyst computer device,wherein the interactive query node tree displays comprise an interactivequery node tree display that displays an interactive query node tree,wherein: each query node in the interactive query node tree correspondsto a separate query designed to assess risk for the first transactionand wherein each query node comprises a hyperlink; upon the analystactivating the hyperlink for a first query node in the interactive querynode tree display, a corresponding query for first query node isdisplayed for the analyst in a first query display, wherein the firstquery display further comprises: means for the analyst to enter aresponse to the first query; an evidence field for the analyst to cite acitation in the one or more transactional documents that supports theresponse to the first query; a suggestion field suggesting one or moreplaces in the one or more transactional documents for the analyst toreview to determine the response to the first query; and a next queryselection button that, when activated by the analyst, causes a secondquery display to be displayed to the analyst, wherein the query for thesecond query display depends on the response by the analyst to the firstquery, and the second query display comprises: means for the analyst toenter a response to the second query; the evidence field for the analystto cite a citation in the one or more transactional documents thatsupports the response to the second query; the suggestion fieldsuggesting one or more places in the one or more transactional documentsfor the analyst to review to determine the response to the second query;and the next query selection button that, when activated by the analyst,causes a third query display to be displayed to the analyst.
 2. Thecomputer system of claim 1, wherein the back-end computer system isconfigured to compute and display an overall risk score for the firsttransaction based on the analyst’s responses to queries in the querynode tree, and wherein the back-end computer system further comprises adocument scoring module for identifying the one or more places in theone or more transactional documents for the first transaction to displayin the suggestion fields for first and second query nodes, wherein thedocument scoring module comprises: a document similarity comparisonmodule for: comparing a first passage responsive to the first query nodeof the one or more transactional documents for the first transaction toa first passage in a second transactional document stored in thetransaction document database for a second transaction, wherein thefirst passage in the second transactional document is responsive to thefirst query node for the second transaction and the second transactionis a similar type of transaction to the first transaction, to identifythe one or more places in the one or more transactional documents todisplay in the suggestion field for the first query node; and comparinga second passage responsive to the second query node of the one or moretransactional documents to a second passage in the second transactionaldocument, wherein the second passage in the second transactionaldocument is responsive to the second query node for the secondtransaction, to identify the one or more places in the one or moretransactional documents for the first transaction to display in thesuggestion field for the second query node, wherein the documentsimilarity comparison module uses cosine similarity scores to comparethe passages of the first and second transactional documents.
 3. Thecomputer system of claim 2, wherein the query database stores citationsof the analyst in the evidence fields for the first and second querynodes to store as possible suggestions in the suggestion fields for thefirst and second query nodes for assessing transaction risk of a futuretransaction with the computer system.
 4. The computer system of claim 3,wherein: the means for the analyst to enter the response to the firstquery provides a suggested response to the first query upon adetermination by the document similarity comparison module that asimilarity score for the first passage responsive to the first querynode of the one or more transactional documents for the firsttransaction to the first passage in the second transactional documentexceeds a threshold similarly score for the first query node; and themeans for the analyst to enter the response to the second query providesa suggested response to the second query upon a determination by thedocument similarity comparison module that a similarity score for thesecond passage responsive to the second query node of the one or moretransactional documents for the first transaction to the second passagein the second transactional document exceeds a threshold similarly scorefor the second query node.
 5. The computer system of claim 4, furthercomprising an administrator computer device that is in communicationwith the back-end computer system, wherein the administrator computerdevice is for displaying administrator displays provided by the back-endserver of the back-end computer system, wherein the administratordisplays comprise user interfaces through which an administratorspecifies the queries for each query node of the query node tree and anassociated query score for possible responses for each query node,wherein the back-end computer system is configured to compute theoverall risk score based on the associated query scores for theresponses provided by the analyst to the queries.
 6. The computer systemof claim 1, wherein the query node tree specifies a progression of querynodes.
 7. The computer system of claim 2, wherein the first querydisplay includes an additional field through which the analyst ispermitted to flag that an issue related to the first query is importantto the risk assessment.
 8. The computer system of claim 7, wherein theadditional field further permits the analyst to enter an importancescore for the first query.
 9. The computer system of claim 8, whereinthe back-end computer system is configured to generate a final riskassessment for the first transaction, wherein the final risk assessmentcomprises the overall risk score for the first transaction and a list ofissues flagged by the analyst as important to the risk assessment.
 10. Amethod of monitoring one or more transactional documents of a firsttransaction for changing risk profiles relative to prior transactions ofa similar type, the method comprising: storing, in a transactiondocument database of a back-end computer system, the one or moretransactional documents of the first transaction in word-searchableform, wherein storing the one or more transactional documents comprisesconverting transactional documents that are not word-searchable toword-searchable form through optical character recognition (OCR),wherein converting the transactional documents to word-search formcomprises identifying individual characters in the transactionaldocuments and identifying equivalent plain text characters for theidentified individual characters using matrix matching and/or featureextraction; storing, in a query database of the back-end computersystem, pre-determined queries for an analyst to investigate in the oneor more transactional documents for the first transaction, wherein forat least some of the pre-determined queries, the query database alsostores corresponding suggestions in the one or more transactiondocuments for the analyst to review to respond to the query, wherein thesuggestions are based on prior reviews of transactional documents forprior transactions that are similar to the first transaction; andserving, by a web-server of the back-end computer system, interactivedisplays to an analyst computer device that is in communication with theback-end computer system, wherein the interactive displays are fordisplay by the analyst computer device, wherein the interactive displayscomprise an interactive query node tree display that display aninteractive query node tree, wherein: each query node in the interactivequery node tree corresponds to a separate query designed to assess riskfor the first transaction and wherein each query node comprises ahyperlink; upon the analyst activating the hyperlink for a first querynode in the interactive query node tree display, a corresponding queryfor first query node is displayed in a first query display, wherein thefirst query display further comprises: means for the analyst to enter aresponse to the first query; an evidence field for the analyst to cite acitation in the one or more transactional documents that supports theresponse to the first query; a suggestion field suggesting one or moreplaces in the one or more transactional documents for the analyst toreview to determine the response to the first query; and a next queryselection button that, when activated by the analyst, cause causes asecond query display to be displayed to the analyst, wherein the queryfor the second query display depends on the response by the analyst tothe first query, and the second query display comprises: means for theanalyst to enter a response to the second query; the evidence field forthe analyst to cite a citation in the one or more transactionaldocuments that supports the response to the second query; the suggestionfield suggesting one or more places in the one or more transactionaldocuments for the analyst to review to determine the response to thesecond query; and the next query selection button that, when activatedby the analyst, causes a third query display to be displayed to theanalyst; identifying, by a document scoring module of the back-endcomputer system, the one or more places in the one or more transactionaldocuments for the first transaction to display in the suggestion fieldsfor first and second query nodes, wherein identifying the one or moreplaces comprises, with a document similarity comparison module of thedocument scoring module: comparing a first passage responsive to thefirst query node of the one or more transactional documents for thefirst transaction to a first passage in a second transactional documentstored in the transaction document database for a second transaction,wherein the first passage in the second transactional document isresponsive to the first query node for the second transaction and thesecond transaction is a similar type of transaction to the firsttransaction, to identify the one or more places in the one or moretransactional documents to display in the suggestion field for the firstquery node; and comparing a second passage responsive to the secondquery node of the one or more transactional documents to a secondpassage in the second transactional document, wherein the second passagein the second transactional document is responsive to the second querynode for the second transaction, to identify the one or more places inthe one or more transactional documents for the first transaction todisplay in the suggestion field for the second query node, wherein: thedocument similarity comparison module uses cosine similarity scores tocompare the passages of the first and second transactional documents;the query database stores the analyst’s citations in the evidence fieldsfor the first and second query nodes to store as possible suggestions inthe suggestion fields for the first and second query nodes for assessingtransaction risk of a future transaction with the computer system; themeans for the analyst to enter the response to the first query providesa suggested response to the first query upon a determination by thedocument similarity comparison module that a similarity score for thefirst passage responsive to the first query node of the one or moretransactional documents for the first transaction to the first passagein the second transactional document exceeds a threshold similarly scorefor the first query node; and the means for the analyst to enter theresponse to the second query provides a suggested response to the secondquery upon a determination by the document similarity comparison modulethat a similarity score for the second passage responsive to the secondquery node of the one or more transactional documents for the firsttransaction to the second passage in the second transactional documentexceeds a threshold similarly score for the second query node.
 11. Themethod of claim 10, further comprising: computing, by the back-endcomputer system, an overall risk score for the first transaction basedon the analyst’s responses to queries in the query node tree; anddisplaying, by an administrator computer device that is in communicationwith the back-end computer system, the overall risk score.
 12. Thecomputer system of claim 11, further comprising displaying, by theadministrator computer device, administrator displays provided by theweb server of the back-end computer system, wherein the administratordisplays comprise user interfaces through which an administratorspecifies the queries for each query node of the query node tree and anassociated query score for possible responses for each query node,wherein the back-end computer system is configured to compute theoverall risk score based on the associated query scores for theresponses provided by the analyst to the queries.
 13. The computersystem of claim 10, wherein the query node tree specifies a progressionof query nodes.
 14. The computer system of claim 10, wherein the firstquery display includes an additional field through which the analyst ispermitted to flag that an issue related to the first query is importantto the risk assessment.
 15. The computer system of claim 7, wherein theadditional field further permits the analyst to enter an importancescore for the first query.
 16. The computer system of claim 15, furthercomprising generating, by the back-end computer system, a final riskassessment for the first transaction, wherein the final risk assessmentcomprises the overall risk score for the first transaction and a list ofissues flagged by the analyst as important to the risk assessment.